METHOD, APPARATUS AND RECORDING MEDIUM FOR IMAGE PROCESSING 

BACKGROUND OF THE INVENTION 
5 Field of the Invention 

The present invention relates to an image processing 
method and an image processing apparatus for changing a 
color and a tone in a specific area included in a color image. 
The present invention also relates to a computer - readable 
10 recording medium storing a program to cause a computer to 

SJ execute the image processing method. 

H Description of the Related Art 

£ Reproduction of digital image data obtained by digital 

H still cameras (hereinafter referred to as a digital camera) 

ryl5 and digital image data obtained by reading images recorded 

Sj on film has been carried out to generate hard copies such 

O as prints, or soft copies on display screens. In such 

reproduction of digital image data, a reproduced image is 
expected to have as high a quality as a photograph printed 
20 from a film negative. 

Color reproducibility of the same person's face can 
vary between prints. Furthermore, in an image including a 
plurality of people, color reproducibility of each person's 
face can be different, depending on the intensity or 
25 position of a light source. In the case of an image 

including a person obtained by photographing, the face of 
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the person attracts the most attention. In order to 
generate a high-quality photograph, correction of the face 
so as to have an adequate skin color is necessary. To 
accomplish this, various methods of changing a skin color 
5 of an image to a preferable skin color, for example by using 

a display means such as a monitor and a mouse have been 
developed- One such method comprises the steps of using a 
mouse to extract pixels of skin color from an image including 
a person's face and changing the skin color value to a color 
^10 and tone of a predefined preferable skin color value using 

the extracted color value as reference (Japanese Patent No. 
^ 2878855, for example) . A second proposed method comprises 

'2 the steps of specifying an area of color to be changed 

contained within a displayed image, specifying the color 
^15 nearest to the most preferable color for the area in which 

-y, the change is desired, upon which the color of the area is 

™ changed to a pre-defined color (Japanese Unexamined Patent 

Publication No. 8 ( 1 9 9 6 ) - 2 2 1 5 4 6 ) . Furthermore, in a third 
proposed method, the areas containing human faces are 
20 automatically extracted and the color of the areas 

containing faces is changed to a predefined color. 

However, in all the methods described above, because 
the skin color of a human face is changed to a predetermined 
skin color, the human face does not necessarily have a 
25 preferable skin color after the change. Furthermore, since 

only the face color is changed, a sharp - contrast face of 
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a person in an image obtained by photographing outdoors in 
sunny weather cannot be corrected. 

SUMMARY OF THE INVENTION 
The present invention has been conceived in 
5 consideration of the above described problems. An object 

of the present invention is therefore to provide an image 
processing method and an image processing apparatus which 
can change a specific area in an image such as a human face 
to a preferable color and tone (hereinafter, the color and 
QLO tone are together referred to as the color-tone), and to 

SI provide a computer- readable recording medium storing a 

S program to cause a computer to execute the image processing 

=p method. 

s An image processing method of the present invention 

115 comprises the steps of: 

receiving selection of a target image having a desired 
color-tone from a plurality of reference images each having 
a different color-tone; 

receiving specification of an area in an image 
20 displayed; and 

changing a color- tone of a desired area including the 
specified area to the color- tone of the target image. 

The "area" refers to an area whose color-tone is to 
be changed in the displayed image. More specifically, the 
25 area refers to an area including an image of an object such 

as a human face, the sky, or green foliage. 
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The "desired area including the specified area" can 
refer to not only the entire specified area but also a 
specific area in the specified area, the entire image 
including the specified area, or areas comprising the 
5 specified area and another area. 

The selection of the target image may precede the 
specification of the area, or vice versa. 

"Reference images" refer to sample images of an object 
whose color-tone has been changed, such as a human face, 
QlO the sky, or green leaves, for example. The plurality of 

N reference images each having a different color- tone are 

S provided so as to enable a user to select the desired 

£ color-tone therefrom. One of the reference images having 

5 the color-tone desired by the user is selected as the target 

ryi5 image. 

The reference images may be generated in advance 
according to a predetermined image of a user's preference. 
For a case in which the reference images are images of human 
faces, for example, image data containing an image of the 
20 face of a user's favorite celebrity, for example, obtained 

by photographing using a digital camera, by reading a film 
or an image with a scanner, or by other means is then 
subjected to processing causing various changes in the color 
and tone of the image of the face, and thereby generating 
25 the plurality of reference images each having a different 

color and tone . 
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An image processing apparatus of the present invention 
comprises : 

storage means for storing a plurality of reference 
images each having a different color-tone; 
5 display means for displaying an image and the 

reference images; 

selection means for selecting from among the reference 
images a target image having a desired color-tone; 

area specification means for specifying an area in the 
Qo image displayed on the display means; and 

^-J conversion means for changing a color- tone of a 

M desired area including the specified area to the color- 

n H 

tone of the target image, 
s The image processing apparatus of the present 

TULS invention may further comprise reference image generation 

PJ 

means for generating the plurality of reference images based 

^3 

Q on a predetermined image of a user's preference. 

The image processing method of the present invention 
may be provided as a program stored in a compu t er - readabl e 
20 recording medium to cause a computer to execute the image 

processing method. 

According to the present invention, the target image 
having a desired color-tone is selected from among the 
reference images, which each have a different color- tone, 
25 and an area in which the color-tone is to be changed is 

specified in the displayed image. The color-tone of the 
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desired area including the specified area is then changed 
to the color-tone of the target image. Thereby, the desired 
area including the specified area in the image can be changed 
to have the color-tone desired by the user. 
5 Furthermore, by generating the reference images based 

on a predetermined image such as an image of a user's 
preference, selection of the target image is made enjoyable. 
BRIEF DESCRIPTION OF THE DRAWINGS 
Figure 1 is a block diagram showing a configuration 
QO of an image processing apparatus according to an embodiment 

Sj of the present invention; 

W Figure 2 shows reference images; 

=p Figure 3 shows an image displayed on a monitor; 

a Figure 4 shows an image in which an area Rl has been 

rd5 specified; 

ry 

S| Figure 5 is a diagram showing a skin-color range in 

O a CIELAB color space; 

Figure 6 shows cumulative histograms; 

Figure 7 is a flow chart showing operation of the 
2 0 embodiment; and 

Figure 8 is a block diagram showing a configuration 
of an image processing apparatus comprising reference image 
generation means. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 
25 Hereinafter, an embodiment of the present invention 

will be explained with reference to the accompanying 
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drawings . 

Figure 1 is a block diagram showing a configuration 
of an image processing apparatus according to an embodiment 
of the present invention. The image processing apparatus 
5 obtains processed image data SI by carrying out image 

processing on image data S obtained by photographing using 
a digital camera, reading an image with a scanner, or another 
means. As shown in Figure 1, the image processing apparatus 
comprises reading means 1 for reading the image data S from 
Q.0 a recording medium or for reading the image data S 

N transferred via a network, image processing means 2 for 

y=J carrying out the image processing on the image data S, a 

=F monitor 3 for displaying the image data S, input means 4 

= comprising a mouse and a keyboard for carrying out various 

nJl5 input operations to the image processing means 2, and 

£ Li 

N Storage means 5 for storing reference image data 

O representing a plurality of reference images which will be 

explained later. In this embodiment, an image represented 
by the image data S includes a human face, and a color- 
20 tone of the face in the image is changed to a preferable 

color- tone . 

The storage means 5 stores the reference image data 
representing the reference images. Figure 2 shows the 
reference images represented by the reference image data. 
25 As shown in Figure 2, the reference image data comprises 

data Ql representing images of the face of the same person 
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in different colors, and data Q2 representing images of the 
face of the same person in different tones set for the 
respective face images represented by the data Ql . The data 
Ql and Q2 are in a hierarchical structure. The images of 
the face in different colors are shown on the monitor 3 based 
on the data Ql . A user selects from among the images of the 
face one having a desired color by using the input means 
4. According to the color of the selected image of the face, 
the images of the face in different tones, based on the data 
Q2 are displayed on the monitor 3. The user selects one of 
the images of the face having a desired tone by using the 
input means 4 and the image of the face having the desired 
color and tone is set as a target image. The reference image 
data representing the target image are input to the image 
processing means 2, as target image data Ml. 

The image processing means 2 carries out the image 
processing of the image data S in the following manner. 
First, the image data S is displayed on the monitor 3. 
Figure 3 shows the image displayed on the monitor 3. As 
shown in Figure 3, the image displayed on the monitor 3 
includes a plurality of images of people's faces. The user 
views the image displayed on the monitor 3 and selects a 
face image whose color-tone is to be changed, by using the 
input means 4. Figure 4 shows a state in which the desired 
face image has been selected. As shown in Figure 4, the face 
image is specified by a rectangular area Rl . 
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After the area Rl has been specified in the above manner, 
the image processing means 2 extracts from the area Rl an 
area Rl ' containing pixels within a predefined skin-color 
range. For this extraction, a skin-color range D circled 
5 in a CIELAB color space shown in Figure 5 is predetermined. 

Pixel values of the image contained in the area Rl are 
converted into the CIELAB space, and pixels having values 
within the skin-color range D are extracted as the 
skin-color area Rl ' . 
Cio After the target image has been selected and the 

skin-color area Rl ' in the area Rl has been extracted in 
S the above manner, cumulative histograms of the target image 

=C and the skin-color area Rl ' are generated. Figure 6 shows 

= cumulative histograms HI and H2 of the target image and the 

nJis skin-color area Rl ' , respectively. The farther the 

'""^ horizontal axes of the cumulative histograms go toward the 

O left and toward the right from the origin in the cumulative 

histograms HI and H2 respectively, the larger RGB densities 
become. For cases in which the image data S comprises data 
20 of RGB colors, the cumulative histograms HI and H2 are 

generated for each color. For simplicity, one set of RGB 
data comprising the data of RGB colors is assumed and the 
cumulative histograms HI and H2 are generated based on the 
RGB data. The cumulative histograms HI and H2 represent 
25 characteristics of the color-tone of the target image and 

the skin-color area Rl ' , respectively. Hereinafter, these 
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characteristics are called characteristic quantities. 

After the cumulative histograms HI and H2 have been 
generated in the above manner, color-tone conversion 
processing to cause the color- tone of the skin-color area 
5 Rl ' to become the color-tone of the target image is carried 

out, based on the cumulative histograms HI and H2 . This 
color-tone conversion processing is carried out by relating 
the cumulative histogram HI with the cumulative histogram 
H2 as shown by dashed lines in Figure 6 so as to cause a 
Cio pixel value whose RGB density is PI in the cumulative 

histogram H2 to have an RGB density P2 , thereby changing 

z — 

ffl the color-tone of the skin-color area Rl ' . Let a function 

=S representing the cumulative histogram HI be g(d') and let 

- a function representing the cumulative histogram H2 be f (d) . 

nJls The processing shown by the dashed lines in Figure 6 can 

^ be expressed by Equation (1) below: 

O 

d' = g Mf (d) ) (1) 

If only the color- tone of the skin-color area Rl ' is 
changed, a boundary thereof looks unnatural. Therefore, it 
20 is preferable for the color- tone to be changed gradually 

at the boundary. 

Operation of the first embodiment will be explained 
next. Figure Visa flow chart showing the operation of this 
embodiment. The image data S is read and input to the image 
25 processing means 2 (Step SI) . The reference image data is 

read from the storage means 5 and the reference images are 
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displayed on the monitor 3 (Step S2) . The user selects the 
target image having the desired color- tone from the 
reference images and inputs completion of the selection by 
using the input means 4 (Step S3) . After selection of target 
5 image has been completed, the image data S is displayed on 

the monitor 3 (Step S4). 

The user views the image displayed on the monitor 3 
and specifies the area Rl including the image of the face 
whose color-tone is to be changed to the color-tone of the 
O.0 target image, and inputs completion of the specification 

SI by using the input means 4 (Step S5) . After selection of 

O the area Rl has been completed, the skin-color area Rl ' 

=p included in the area Rl is extracted (Step S6). 

s After the extraction of the skin-color areas Rl ' , the 

hjLS cumulative histograms HI and H2 of the target image and the 

s\ skin-color area Rl ' are generated (Step S7) . Based on the 

O 

Q cumulative histograms HI and H2, the color-tone of the 

skin-color area Rl ' is changed to become the color-tone of 
the target image according to Equation (1), and the 
20 processed image data SI is obtained (Step S8) to end the 

procedure . 

As has been described above, in this embodiment, the 
target image having the desired color- tone is selected and 
the color-tone of the specified area Rl is changed to become 
25 the color-tone of the target image. Thereby, the skin-color 

area Rl ' in the specified area Rl can be changed by the user 
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to have the color-tone desired. 

In the above embodiment, the color- tone of only the 
skin-color area Rl ' in the specified area Rl is changed. 
However, a color-tone of the entire area Rl may be changed. 
Alternatively, a color-tone of a predetermined area 
including the area Rl or the entire image displayed on the 
monitor 3 may be changed. 

In the above embodiment, the area Rl is specified as 
a rectangular area. However, the area may be specified as 
an area having any shape, such as the shape of a human face, 
for exampl e . 

In the above embodiment, characteristic quantities of 
the color-tone of the target image and the skin-color area 
Rl ' in the area Rl are represented by the cumulative 
histograms HI and H2, which is not necessarily limited to 
these cumulative histograms. 

In the above embodiment, the cumulative histogram HI 
of the target image is generated. However, cumulative 
histograms of all the reference images may be generated in 
advance and stored in the storage means 5. 

In the above embodiment, face images of one and the 
same person are used for the reference images. However, a 
plurality of face images of the same person rendered in 
different skin colors based on skin-color characteristics 
determined by race, gender or the like, so that the target 
image can be selected therefrom. 
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In the above embodiment, the color- tone is changed for 
the human face. However, a target image can be selected for 
a color of the sky, green foliage, or the like contained 
in an image, as in the case of the human face described above. 
In this case, reference images in different colors and tones 
for the sky, green foliage, or the like are used and a 
color-tone of an area Rl can be changed to a color-tone of 
a target image. 

In the above embodiment, in selection of the target 
image, the desired color is first selected, and then the 
desired tone is selected. However, it is also possible to 
first select the desired tone, and then select the desired 
color. In this case, the images of a face in different tones 
are displayed on the monitor 3 and the image of the face 
having the desired tone is specified. The images of the face 
in different colors are then displayed on the monitor 3, 
according to the selected tone. 

The image processing apparatus according to this 
embodiment may further comprise reference image generation 
means 6 for generating the reference images, as shown in 
Figure 8. The reference image generation means 6 generates 
the reference image data, and the reference image data is 
stored in the storage means 5. In this case, the reference 
image data is generated in the following manner. If the 
reference images are human faces, image data of the image 
of a face of a user's preference is obtained by photographing 
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using a digital camera, reading a film or an image with a 
scanner, or another means. The image data is input to the 
reference image generation means 6. In the reference image 
generation means 6, a color and a tone of the image data 
are changed in various ways and the reference image data 
in different colors and tones is generated. 

By generating the reference images based on a 
predetermined image such as a user's favorite image, 
selection of the target image is made enjoyable. 

In addition, all of the contents of Japanese Patent 
Application Nos . 1 1 ( 1 9 9 9 ) - 3 6 9 3 3 9 and 2000- 329052 are 
incorporated into this specification by reference. 
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